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Abstract 

We study the deviation of yukawa coupling in the gauge-Higgs unification scenario 
from the Standard Model one. Taking into account the brane mass terms necessary for 
generating the flavor mixing and removing the exotic massless fermions, we derive an 
analytic formula determining the KK mass spectrum and yukawa coupling. Applying the 
obtained results to the tau and bottom yukawa couplings, we numerically calculate the 
ratio of the yukawa couplings in the gauge-Higgs unification and in the Standard Model. 


1 Introduction 


Although a Higgs boson was discovered at the CERN Large Hadron Collider (LHC) 
experiment [1], the couplings of the Higgs boson to the Standard Model (SM) fields and 
the self-couplings of Higgs boson have not been precisely measured. It is therefore still 
an unsettled issue whether the Higgs boson is the SM one or that of physics beyond the 
SM. Physics beyond the SM is expected to exist by several reasons such as the hierarchy 
problem. 

Gauge-Higgs unification (GHU) [2] is one of the attractive scenarios beyond the SM, 
which provides a possible solution to the hierarchy problem without supersymmetry [3]. 
In this scenario, the SM Higgs boson and the gauge fields are unified into the higher 
dimensional gauge fields, i.e. Higgs boson is identified with extra spatial components 
of higher dimensional gauge fields. A remarkable fact is that the quantum correction to 
Higgs mass (and potential) is UV-finite and calculable due to the higher dimensional gauge 
symmetry though the theory is the non-renormalizable. The finiteness of the Higgs mass 
has been studied by explicit diagrammatic calculations and verified in models with various 
types of compactification at one-loop level [4] and even at the two loop level [5]. The 
finiteness of other physical observables such as S and T parameters [6], Higgs couplings 
to digluons, diphotons [7], Muon g — 2 and the EDM of neutron [8] have been investigated 
by the present authors or one of them. 

The fact that the Higgs boson is a part of gauge fields implies that Higgs interactions 
are governed by gauge principle and may provide specific predictions in LHC physics. 
From this viewpoint, the diphoton and Zy decay of the SM Higgs boson produced via 
the gluon fusion in the framework of gauge-Higgs unification was studied and remarkable 
predictions were obtained [9]. In order to explain experimental results of diphoton decay 
and 126 GeV Higgs boson mass, some extra matters are required and they may predict a 
possible dark matter candidate. It has been also shown that the Zy decay is not affected 
at one-loop level, which is a distinctive prediction uncommon in other models of physics 
beyond the SM. Thus, Zy decay is considered to be a good probe of GHU. 

In this paper, we focus on the fermion coupling of Higgs boson in the GHU scenario, 
i.e. Yukawa coupling, whose measurement in future would be very important to clarify 
the origin of the Higgs field. In GHU scenario, yukawa coupling generically deviates from 
the SM one as a consequence of the Higgs boson as a gauge field. Let us parametrize the 
fermion mass term as 


m(v)if)i/}, (1.1) 

where m(v) is a mass function of the vacuum expectation value (VEV) of Higgs field. 
Physical Higgs coupling to fermions are obtained by expanding the Higgs field around its 
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VEV v, 
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m(v + h H- -hi/ii/i + 
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( 1 . 2 ) 


where h is a physical Higgs held and the second term is its coupling to fermions of our 
interest in this paper. Its coefficient / = d?) ^’ > is yukawa coupling. • • • implies higher 
order terms in h which are irrelevant throughout this paper. In GHU, the Higgs held 
is a zero mode of gauge held A ^ for hve dimensional case. If the fifth dimension y 
is compactihed on a circle S 1 with a radius R , a constant A^ cannot be removed by 
the gauge transformation and has a physical meaning of Aharanov-Bohm (AB) phase or 
Wilson-loop as 


W 


Pexp 



exp [ig i wRAf ) } , 


(1.3) 


where g,g^ are 5D and 4D gauge couplings, respectively. An important point of eq. (1.3) 
is that W is periodic with respect to A^ under Ay —» Ay + 2 /(g^R). This fact is one 
of the characteristic features of GHU and the physical observables are expected to have 
the periodicity in the Higgs held. Actually, it has been already reported in [10] in the hat 
extra dimensional case and [11] in the warped extra dimensional case that Higgs coupling 
to the fermions have such a periodicity. In [10], the ratio of yukawa coupling of GHU and 
the SM one is derived as 


^? HU ~ ^-vnRcot (^rvn r) , (1.4) 

/SM 2 V 2 / 

which is quite distinctive from the other models beyond the SM. In particular, as was 
pointed out in [10,11], the yukawa coupling vanishes at v — 1 /(g^R) due to the periodicity. 

In the minimal supersymmetric standard model (MSSM) case, the corresponding ratio 
is known to be 


/mssm f (up - type quarks), 

/ SM (down — type quarks and charged leptons), 

where a is a mixing angle of two Higgs doublets in the MSSM and /3 is defined as tan /? = 
(H u )/ (Hd). As for the UED models, the yukawa coupling is the same as the SM one. 
Thus, it is a very important issue for the new physics search to measure the yukawa 
couplings precisely at LHC and ILC. 

We study the deviations of Higgs coupling to fermions from the SM predictions by 
using a five dimensional GHU model of SU(3) gauge theory. Unlike the analysis where 
only the bulk Lagrangian was considered [10], we take into account the effects from the 
brane mass terms necessary in a more realistic model for generating the flavor mixing as 
clarified in [12] and removing the exotic massless fermions absent in the SM. The brane 
mass terms change the boundary conditions of mode equations, which give the formula to 


2 










determine Kaluza-Klein (KK) mass spectrum m n (v). Therefore, it is crucial for the study 
of the deviation of Yukawa coupling in a realistic model to incorporate the appropriate 
brane mass terms. Solving the mode equation with the correct boundary conditions, we 
will derive analytic formulas of determining KK mass spectrum and the ratio of Yukawa 
coupling in GHU and the SM one. As an illustration, we numerically calculate its ratio 
for the tau and the bottom yukawa couplings, which are expected to be more promising 
detectable couplings in the quark and lepton sector comparing to those of other lighter 
fermions. 

This paper is organized as follows. In section 2, we introduce our model. We elaborate 
the equations of motion and the corresponding boundary conditions. Analytic formulas 
determining KK mass spectrum and the expression of yukawa coupling are derived in 
section 3. Numerical calculations for the ratio of the tau and the bottom yukawa couplings 
in GHU and the SM ones are also performed as an application. Section 4 is devoted to 
summary. In appendix A, the derivation of the equations of motion is described in detail. 
The validity of the analytic formula obtained in this paper is checked by taking various 
limits of parameters in appendix B. 


2 The Model 


In this section, we introduce our model. We consider an 577(3) gauge theory in a five 
dimensional space-time where an extra dimension is compactified on an orbifold S 1 /Z 2 . 
The Lagrangian consists of two parts; 


£ 


£ 


Bulk 


£ 


Brane• 


( 2 . 1 ) 


One is the Lagrangian in the bulk where the extra dimensions spread and the other is 
that on the brane located at fixed points y = 0, nR. The bulk Lagrangian is 

£ Bu ik =$(3)(i - Me(y))^{ 3) + $(6*)(i ^ 6 * - Me(y))^(6*) - ^Tr F MN F MN , (2.2) 

where the 3 and the 6 stands for the three and six dimensional representations of 517(3), 
the covariant derivative is given by the 517(3) gauge field p = Y M (8 m — igA a M T a ) with 
the appropriate generators T a for the corresponding representation, and the field strength 
of 51/(3) is denoted by F MN . The components of 3 and 6* are 
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,-0(6*) = 
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The r field belongs to different isospin but has the same electric charge as r lepton, so 
that they will mix each other by the VEV of A y . This matter contents are common in 
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quarks and leptons except for the top quark. The fermion mass is at most of order W- 
boson mass and the top quark has a mass around twice of the W-boson mass, so it has to 
be embedded in higher dimensional representation such as 15 for example to obtain the 
enhancement factor “2”. 

Since the fifth dimension y is compactihed on the circle S 1 , the periodic boundary 
condition is imposed for the fields 

<f>i{y) = My + 2nR )- ( 2 - 4 ) 

To have the chiral theory, the Z 2 symmetry is imposed on the fermions. By assigning 
even eigenvalue of Z 2 to the right-handed singlet and left-handed doublets for the SM 
fermions, their zero modes remains massless. For the singlet lepton 0, the Z 2 eigenvalues 
are assigned as 

0l(+2/) = -0l(-2/),0r(+ 2/) = 0fl(-y). (2.5) 

For the lepton doublets x, they becomes 

Xl(+v) = XL(-y), xr(+v) = ~xr(-v ), (2-6) 

where the chiral projection operators are defined as L — and R = For other 

cases, the Z 2 parities are given by 

1>(+y) = 7 5 V>(-2/), x(+y) = -7 5 x(-y)- (2.7) 

We note that the SU(3) gauge symmetry is simultaneously broken to SU( 2) x U{l)y and 
the SM Higgs doublet is realized in Ay }} by the appropriate Z 2 parity assignment. 

In general, we have extra massless fermions which are not included in the SM. In par¬ 
ticular, the two massless SU (2) doublets appear per generation since up and down sector 
fermions should be embedded in different representations. One of the linear combination 
of them corresponds to the SM SU( 2) doublet, but the other orthogonal one should be 
removed from the low-energy effective theory. A possible way is that they couple with the 
brane-localized fermions (r^ and Tg) and become massive through the Dirac mass terms. 

^Brane V7rRT^M B Tu5(y) + \fnRM B T^8(y) + h.c., (2.8) 

where the is the massive tau leptons orthogonal to the massless tau lepton tsm- They 
are mixing states of the r and r' which are defined by 


( T0 


/I 0 0 0\ 


( T X \ 



r 

t' 

w 

= 

0 cos 6 sin 6 0 

0 — sin 9 cos 9 0 
\0 0 0 \j 


tsm 

Th 

\r 3 J 

= U 

tsm 

th 

V T-3 / 


These mixings play an important role to produce the flavor mixings [12]. 
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To achieve our purposes, we concentrate on the tau leptons and the equation of motion 
(EOM) derived from the lagrangian as follows; 



drf + i^-v^-Meiy) 


T 


id„rfT^8{y) 

id^T^Siy) 


V ttRM b 


/ 0 \ 

0 

Tb 

VbJ 

\fnRM B r B 8(y), 

VirRM B T 3 8(y), 


<%), 


( 2 . 10 ) 

( 2 . 11 ) 

( 2 . 12 ) 


where r = (ti, tsm, Th, T 3 ) t . Si is defined by 


Si = U ] 




(2.13) 


where U connects the basis between (r, r') and (th,tsm) as we mentioned before, ny is a 
Pauli matrix. The factor y/2 comes from the group theoretical factor of the representation 
of 6* (two-rank symmetric tensor). 

Defining r to eliminate Si as 


then we have 


r = exp 


• CJ 

*y*>E 3 y 


T, 


\id„Y - dy'f - Me(yj] f 

i9^r B 5(y) 

id^r B S(y) 


VttRMb 


/ 0 \ 

0 

Vb/ 
\fnRM B T B 8(y), 

\fnRM B T 3 8(y). 


%)> 


(2.14) 


(2.15) 

(2.16) 
(2.17) 


The boundary conditions (B.C.s) of the f at y = 0 is same as before, but it changes at 
y = 7T R because of the Wilson line phases. We summarize the Z 2 conditions on the f at 
the origin y = 0 

H-y) = p l5 r(y), ( 2 . 18 ) 

where P = diag(—, +, +, —) and the periodic B.C. at y — \ttR\ with respect to S 1 


exp 


■ 94 v ' 
L *yUSl|/ 


y=7rR 


= exp 


■ 9 4 v 
L *yUSi|/ 


y=-irR 


In other words, these periodicities are rewritten in terms of the parities at the y 
namely, 


J T-(y) 


odd 


= 0 


y=nR 


(2.19) 
= ttR, 

( 2 . 20 ) 
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where [• • • ] 0 dd/even stands for extracting odd/even function of y. The conditions of the 
derivative d y f are obtained by integrating the EOM around y = nR. Then we have the 
following two conditions from (2.19); 


f [e l9 ^ iy f(y)] odd \ y=nR = 0, 
| ¥^ lV ^ d y^y)]oJy=,R 


-M [< 


e 2 


'( 2 /)] 


even I y=irR 


( 2 . 21 ) 


A few comments are listed. We omit the strong interaction through this paper since 
our purpose is to investigate the effects of the flavor mixing and brane mass term on the 
deviation of yukawa coupling from the SM one and the strong interaction does not affect 
the deviation. 

The Weinberg’s angle in this model is not consistent with the observed one, which 
is obtained by introducing an extra U(l)' gauge group or the brane localized kinetic 
terms. This does not affect the deviation of yukawa couplings originated from an SU (2) 
gauge coupling and we can safely ignore them in this paper. By adjusting these extra 
U(l)' charge, the hypercharge can be changed as we like. Therefore, the r lepton and 
the b quark can be assigned to the same representations 3 and 6* by taking different 
hypercharges [13]. 


3 Deviation of Yukawa Coupling in GHU from the 
SM one 

In this section, we discuss the deviation of yukawa coupling in GHU from the SM one. 
First of all, we derive the analytic formula determining the KK mass spectrum from the 
boundary conditions of fermions, the continuous conditions at y = \ttR\ and the Z 2 con¬ 
dition. Next we obtain the yukawa coupling in GHU through the analytic formula by 
differentiating the KK fermion mass m(y ) with respect to the VEV v. As a phenomeno¬ 
logical application, numerical calculations for the ratio of the tau and the bottom yukawa 
couplings in GHU and the SM ones are performed. 


3.1 Analytic formula determining KK mass spectrum 


To begin with, we expand the t in terms of the mode functions as follows; 


T(x,y) = 


71=0 


X 1L ( x ) f[ n L (: V) + (x) /S ( y) 

*~SMl( X ) fsML(y) + X SMr( X ) fsMR.{y) 

( x ) fni (y) + x hr ( x ) fnR ( y ) 

X 3L (X) fsL (y) + X 3R ( x ) f3R iv) . 


(3.1) 
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Hereafter, we omit the index n in the mode functions for notational simplicity. Then the 
eigen equations of the mode functions of f\ is obtained as 


m n fi L + (d y - Me(y))fi R = 0, 
m n fiR + (~d y - Me(y))fi L = 0, 


(3.2) 


where m n is a KK mass eigenvalue. The mode functions of tsm obey the same eigen equa¬ 
tions. The eigen equations for mode functions of fn, are given by using the integration 
by parts of the delta function f dyf(y)d v S(y) = — f d yd v f(y)5(y). 


(3.3) 


[m 2 n -dl~ 2 M(S{y) - S(y - tt R)) + M 2 ]/hl = - 1 rRM^f HL 8(y), 

[m 2 n -d1 + 2 M(S(y) - S(y - 1 tR)) + M 2 ]f UR = -nRM£f nR 5(y), 

for fn-, and the mode functions of % obeys the same eigen equations. 1 

The eigen equations are immediately solved by respecting the Z 2 parties at the origin 
(2.21) as 

' fiL oc sin (y/ml - M 2 y), f 1R oc cos (^m 2 n - M 2 \y\ + «i), 

fsML OC cos (\/ml - M 2 \y\ + a SM ), /smr oc sin (y/m 2 n - M 2 y), 
fuL oc cos (^/m 2 n - M 2 \y\ + a H ), fun oc sin iy/m* - M 2 y ), 

J 3L oc 8m(y/m*- M 2 y), f 3R oc cos (^m 2 n - M 2 \y\ + a 3 ), 

where the a’s in the above argument are defined as 

y/ ml — M 2 . M 


(3.4) 


COSCKi = 


COS Q(gM — 


tan ctjj — 


tana's = 


mn _ 

V ml - M 2 

m n 

2 M - uRM 2 

2 ^m 2 n - M 2 ’ 
- 2 M - 7 1 RM 2 


smai = — - 


m ri 


sin asM = -- 


M 


m r . 


(3.5) 


2 ^m 2 n -M 2 ‘ 

The brane mass terms Mb are considered to come from the underlying theory, such as a 
Grand Unified Theory, it is therefore much larger than the compactihcation scale. Then 
we take the limit Mb —> 00 and it reduces to «h = «3 = —7t/2. 

To obtain the practical B.C.s leading to the KK eigenstates, we first calculate the 
phase matrix. 


• 94V V 

q i —^iy — 


(U) (1,2) 
( 2 , 1 ) ( 2 , 2 ) 


(3.6) 


The submatrices become 


( 1 , 1 ) = 


1 + cr 3 1 -03 


cos 2 9 


g±vy 1 - 03 . 2 g^vy g A vy 

cos-1-sm 9 cos — -=r + vj 1 cos 9 sin-. 

2 2 s/2 


(3.7) 


lr The derivation of these mode equations are described in Appendix A. 
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fuvy 


n - ■ a of 94vy 9 iVy\ .1 + a 3 . g A vy .I -03 . 

1,2 —a sm 0 cos 6 cos —=— cos- — i -sin 0 sin-b % -sm 6 

v ’ ’ \ V2 2 y 2 2 2 

, n in +■ a a( 94Vy 9iVy\ -1 + ^3 . a . 

(2, 1) =cr sin 0 cos 0 cos —■=— cos- — l -Sill 0 SI 

' V Vi 2 ) 2 

( 2 , 2 ) 


Sill 


V2’ 

(3.8) 

. 9ivy . . 1-^3 . n . 94Vy 

sm —-b i —-— sm 0 sm — 

2 2 V 2 


(3.9) 


1 + a 3 2 „ ___ 9ivy 


sin“ 9 cos --b 

2 2 


1 - cr 3 1 + cr 3 2 
-1-cos 0 


muy • „ . 94^9 

COS r + %o i COS 0 Sill , 

V 2 v 2 


(3.10) 


where cti, 2,3 are the Pauli matrices and cr ± = ai . 
from the continuous condition ( 2 . 21 ) [e*^ _ySl r(i/)]odd 
odd function from each part. 


The B.C.s on the left-handed part 
= 0 is given by extracting the 

y=jrR 


o = 

D 




i; r(y)} 


cos |/il 


2- 

0 


odd 

y=-nR 

i COS 0 Sill f/sML 
i sin 0 sin ^/sml 


sin 0 sin |/hl 0 \ ,(n) 

i cos 0 sin ^/hl cos ^/ 3 l y Tl ’ 


(3.11) 


where A = g^vnR. The /il/sml/hl/ 3 L hi the above expressions are understood to be the 
values of mode functions at y = ttR. 

Next we discuss the conditions on the derivatives of mode function in eq. (2.21) 


• 941 
1 2 


^ iy lbd y T{y) 


- odd 


= -M 


y=nR 


•941 
1 2 


Si y 


T (y) 


y=nR 


(3.12) 


The conditions on the derivatives of mode functions are obtained from the lower part of 
eq. ( 2 . 21 ) as 


0 = 

i cos 0 sin ^(d y + M)f lL (cos 2 0 cos | + sin 2 0 cos -^=)(d y + M)/ S ml 
—i sin 0 sin ^(d y + M)f 1L sin 0 cos 0(cos ^ - cos \){d y + M)f SML 

sin 0 cos 0(cos ^ — cos |)(<9 y + M)/hl i sin 9 sin ^(d y + M)f 3L „ (ri) 
(sin 2 0 cos | + cos 2 0cos^)(9 J/ + M)/ H l i cos 0 sin ^(^ + M)/ 3L TjL 

(3.13) 


Note that the derivatives of the mode functions in the above are also understood to be 
d vf = dyf\ y =„R- 

Combining these two conditions (3.11) and (3.13), we have 

cos |/il 

6 

i cos 0 sin \{d y + M)f 1L 
—i sin 0 sin |( d y + M) f\ L 
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i cos 0 sin |/ SM l 
i sin 0 sin ^/sml 

(cos 2 0 cos | + sin 2 0 cos ^){d y + M)f SML 
sin 0 cos 0(cos - cos |)(<^ + M)f SM l 


































—i sin 9 sin |/hl 
% cos 9 sin ^/hl 

sin 9 cos 9 (cos — cos |)(<9 y + M)/ H l 
( sin 2 9 cos | + cos 2 9 cos (c\, + M)/ H l 


0 


c °s ^/sl 

i sin 0 sin ^(d y + M)f 3L 
i cos 6» sin ^=( d y + M)f 3L 
(3.14) 



To have non-trivial solutions of r^, the determinant of the above matrix must be van¬ 
ished. Substituting the mode functions (3.4) into the above matrix, the determinant takes 
the following form 


0 


cos ^ sin cj) n i cos 9 sin A cos(0 n — au) 

0 i sin 9 sin cos(0 n — au) 

i cos 9 sin A cos(</> n + a±) —(cos 2 9 cos A + sin 2 9 cos -3=) sin <f) n 
—i sin 9 sin A cos (cj) n + «i) — sin 9 cos 0(cos ^ — cos A) sin cj) n 

—i sin 9 sin A cos(0 n + an) 0 

i cos 6* sin cos(0 n + ctn) cos sin <j) n 

— sin 9 cos 6>(cos ^ — cos |) sin(0 n + a? H + «i) i sin 9 sin cos(0 n + au) 

— (sin 2 0 cos | + cos 2 9 cos ^) sin(0 n + an + «i) i cos 9 sin cos(</> n + a \) 

(3.15) 

sin 2 0 j> cos 2 a i , 

(3.16) 


cos(0 n + «i) sin^ 


• Sill 


A 


A 


A 


+ <; sm - + | sm — - sm - 


where we employ <p n as nR^/w? n — M 2 . The phase «h is here set to be — |. 

Then, we find the three conditions of vanishing determinant as 

{ cos (<j) n + «i) = 0, 

sin (j) n — 0, (3.17) 

sm l ^ = sin 2 A — fsin 2 A — s in 2 -4=) sin 2 6* = sin 2 A C os 2 0 + sin 2 -4= sin 2 6*. 

cos 2 CHI 2\^2 V2 / 2 \/2 

The last condition depending on the Higgs VEV A = g^vnR includes a zero-mode fermion 
since the SM fermions get a mass from the VEV of the Higgs field. Note that the mixing 
parameter 9 appears in the right hand side of the third line in (3.17) due to the mixing of 
r and r'. When 9 = 0, it is expected that the above result reduces to the case where only 
the representation 3 is introduced. In fact, our obtained result agrees with the results 
in [10] as we expected. 


3.2 Deviation of yukawa couplings in GHU from the SM one 

In the above subsection, we have derived an analytic formula determining the KK mass 
spectrum (3.17). The last formula in (3.17) provides the SM fermion spectrum. Although 
it cannot be analytically solved in terms of m n (v), we can still get the exact form of their 
derivatives: -^m n (v). Expressing the last formula in the following way 

M 2 \ 

— pm (3.18) 

mij 
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where 

F( A) = sin 2 ^ f sin 2 ^ - sin 2 sin 2 9, 

it is straightforward to calculate the derivative of m n (v), 

d m n g± (j) n cos an 1 dF(A) 

dn 


(3.19) 


(3.20) 


2 </> n cot (ft n — sin 2 aq F(\) dA 

where eq. (3.17), cosoq = and sin a { are used. 

This is yukawa coupling in GHU what we would like to obtain and it varies according 
to the Higgs VEV v. To compare it with the SM yukawa coupling, we focus on the zero 
mode (n = 0) sector and the ratio of them is found to be 


r dmp 

J di> 


A 


(j) o COS Ot\ 


1 dF(X) 


(3.21) 


/sm yf 2TvRm 0 0o cot </>o — sin 2 ot\ F(X) dA 

For the case M > mo (the zero mode is plausible), we should replace ^>o with inR\JM 2 — tUq, 
and then, we have 

/ A M 2 -m 2 d 


/sm 2 M 2 — nRMrri^^/M 2 — m§ cot\\(irR\JM 2 — mg) dA 

3.3 Numerical study 


ln(F(A)). 


(3.22) 


In this subsection, we apply the above result (3.22) to the tau lepton and the bottom 
quark which would be measured at the LHC or ILC more promising than those of other 
fermions. Moreover, the tau lepton and the bottom quark can be assigned to the same 
fundamental representation of SU(3) in GHU as mentioned above, therefore the result 
(3.22) can be independently applied to the both cases. Regarding the fermion mass as an 
input parameter mo = m T (mb) and rewriting the Higgs VEV by W-boson mass through 
Miy = gv/2, the analytic formula determining the fermion mass and the ratio of the 
yukawa coupling are the following 

M 2 


sinh" 


/ 


71R 


M2 - m l(b) 


m 


r(b) 


m 


( 6 ) 


X 


sin 2 (ttRM w ) — ("sin 2 (nRM w ) — sin 2 (V2ttRM w )) sin 2 9 , (3.23) 


M 2 


m: 


r(6) 


fsM M 2 - 7TRm 2 (b) yjM 2 — m 2 (fe) coth(7r R^M 2 - m 2 (fe) ) 

sin ( 2ixRM w ) — [sin (27 tRM w ) — v^sin (2^2nRM w )~\ sin 2 9 


X 7lRM w 


1 — cos(2nRM w ) — [cos(2y/2 ttRM w ) — cos ( 2ttRM w )~\ sin 2 6 


(3.24) 


There are three parameters R , M and 6 in our theory, but one of them can be determined 
by the eq. (3.23), that is to say, the combination RM is determined to reproduce the 
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Figure 1: The plots of the ratio of the yukawa coupling as a function of the compactih- 
cation scale are shown. The left (right) one is the case for the tau lepton (the bottom 
quark). 

realistic fermion mass. We plot the ratio of the tau and the bottom yukawa coupling with 
some values of 6 as a function of the compactihcation scale Rr 1 in Figure 1. We see that 
the both yukawa coupling in GHU are almost consistent with the SM yukawa coupling, 
which is also consistent with the present experimental data [14]. It is quite natural since 
the weak scale M\y is much smaller than the compactihcation scale. We also note that 
the periodicity of the VEV v in yukawa coupling of GHU exists if and only if 9 — 0, 7t/ 2. 
This is because yukawa coupling of GHU contains different two functions e 2inRM w and 
e 2 V 2 nrRM w w j^} 1 different periodicity in v. 

4 Summary 

In this paper, we have studied deviations of yukawa coupling in GHU from the SM one by 
taking a five dimensional SU(3) GHU model on the orbifold S 1 /Z 2 . It has been already 
pointed out in [10,11], the fermion mass m(y ) is periodic with respect to the Higgs VEV v 
in GHU scenario. Then the derived yukawa coupling can be a nonlinear function of Higgs 
VEV v with fermion bulk mass. In the extreme case, the yukawa coupling vanishes at 
v = 1/(g±R) even though the fermions get nonzero mass from the Higgs VEV v. However, 
it is not clear that these properties are common in such scenario since several modifications 
are needed to construct realistic models. 

What should be emphasized in this paper is that the brane mass terms and the addi¬ 
tional different representations are taken into account unlike the previous work [10]. In a 
realistic model of GHU, the brane mass terms are indispensable not only to remove exotic 
massless fermions absent in the SM but also to realize the flavor mixing as clarified in [12], 
Since the brane mass terms change the boundary conditions for equation of motion and 
this might change KK mass spectrum and yukawa coupling, it is important for our study 
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to incorporate the brane mass terms in realistic model of GHU. 

We have derived an analytic formula determining KK mass spectrum and yukawa 
coupling. It can be shown that the nonlinear yukawa coupling still appears even in 
the case where the brane mass terms are considered. It should be noted that there 
is vanishing points of yukawa coupling in this theory even though the periodicity of 
VEV v is generically lost in the yukawa coupling since it contains two different periodic 
function. This difference comes from the fact that the different kinds of representations 
are introduced to construct more realistic models in GHU. As an application, we have 
numerically studied the ratio of tau and bottom yukawa couplings in GHU and the SM, 
which would be measured at the LHC or ILC more promising than those of other lighter 
fermions. We have found that the both yukawa coupling in GHU are almost consistent 
with the SM yukawa coupling, which is also consistent with the present experimental 
data [14]. It is expected that the consistency is true in any GHU scenario since the Higgs 
VEV v is much smaller than the compactification scale Rr l . We can always expand the 
yukawa couplings in terms of vR and safely neglect the higher order terms of vR, i. e. the 
linear Higgs field approximation of yukawa coupling is always good picture in realistic 
parameter space. 
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A Derivation of mode equations 


In this appendix, we describe the derivation of the mode equation, which is skipped in 
the main text. Note that there are four kinds of fields, but their difference is whether the 
brane mass term exists or not. We first discuss the EOM of the f\ and rg M without the 
brane mass term. Substituting mode expansions of f\ (3.1) into the EOM (2.15), we have 

f id^r^fiL + (d y - Me(y))t 1R f lR = 0, 

[id l rf ll f\ R f\n + (~d y - Me(y))f 1L f 1L = 0. 

Replacing id ^with m n r[ n R / L -, the above mode equations become 


m n fiL + ( d y - Me(y))f 1R = 0, 
m n fiR + (-dy - Me(y))fi L = 0. 


Since the 7 sm obeys the same EOM of the Ti, the corresponding mode functions /sml and 
fsMR obey the same equations. 
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Next, we derive mode equations of th,T3. To eliminate the brane localized fermion 
from the mode equation, we multiply the conjugate of the differential operator of EOM 
from the left-hand side; 


[-id^Y + 0 w 75 - Me(y)][id^ - d y ^ - Me(y)\ f H 
= -yfirRMsl-id^ + d y ^ - Me(y)\r^6(y). (A.3) 


In the right hand side of the above equation, ignoring the last term because of the prop¬ 
erty of the sign function: e(0) = 0 and the integration by parts of the delta function 
f dyf(y)d y 5(y) = — f dyd y f(y)5(y), we arrive at 

['9 2 - d* - 2 M(5(y) - 8(y - ttR))< y 5 + M 2 ]t h = -7r RM^r H 5(y), (A.4) 

and the corresponding mode equation becomes 


[m 2 n - d 2 y - 2 M(6(y) - 5(y - nR)) + M 2 }f UL = -TrRM^f HL 8(y), 
[m 2 n ~d 2 y + 2 M(5(y) - 5(y - irR)) + M 2 )f nR = -nRM%f UR 5(y). 


As mentioned at the beginning of this section, the mode functions fsL and obey the 
same mode equations. 


B Consistency checks of the analytic formula 


In this appendix, we check the consistency of the analytic formula determining KK mass 
spectrum in this model by considering some specific cases. These observations support 
the validity of our analytic formula derived in this paper. 


• Case 1: M, M B —> 0 and 9 = 0 

We first consider the simplest case vanishing the brane mass term and the mixing 
of SU(2) doublets. In this case, the analytic formula determining the KK mass 
spectrum is reduced to 



It indicates that the pattern of the mass spectrum is two kinds: One is ^ ± ^ and 
the other is ^ ± ^=. Due to the fact that the r leptons in the 3 and 6* does not 
mix each other, their form of the KK mass spectrum retain the property of each 
representations. Namely, the r leptons in the 3 and 6* have the yukawa coupling | 
and respectively. 

• Case 2: 9 —> 0, M B —> oo 

Next, we discuss the case where the brane mass term exists. Since the mixing 
parameter 9 is taken to be zero, the r B is equivalent to t' which come from the 6*. 
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Thus it is expected that the zero mode of r' become massive by the brane mass M B . 
In this case, the analytic formula determining the KK mass spectrum is found as 


0 = 


sin 2 (j) n — sin 2 — cos 2 a± 


sin <p n = 0, 

sin (j) n cos(0 n - «i) ( cos(0 n - au) = 0, (B.2) 


sm 2 § = s -^, 

2 cos z a i 7 


where we set a B to be —tt/2. We notice that only the last condition depends on 
the Higgs VEV through A and includes the zero mode fermion mass. For the case 
m n < M, the last condition becomes 


A 

sm — = 
2 


■ sm 71- 


cos ct i 


rVK 


M 2 ) = ±- 


77U 


\[M T ^ 


mi 


-i sinh(7 tR\/M 2 — m%), 


where we replace yj— M 2 with iy/M 2 — m Then we have 

• ^ , m n . . . „ r . , M .A 

sm — ~ ±—— smh(7r RM ) =>■ m n ~ ±————— sm —. 
2 M K sinh(7 tRM) 2 


(B.3) 


(B.4) 


Namely, the yukawa suppressions due to the bulk mass appears. Moreover, the 
A —> 0 recovers massless mode m n = 0. This case reproduces the result in [10]. 


• General case M B —>■ oo: 

Finally, we consider the most general case, the 3 and 6* mix in arbitrary angle 9 
which is discussed in the main text. The mode functions at y = nR in this case 
become 


fiL oc sin 0„, /sml OC cos( 0„ - «i), f UL oc cos(0 n + a H ), 5zl oc sin 0„, (B.5) 

and the derivatives are given as 

” {d y + M)f 1L = m n cos (cj) n + «i), 

(d y + M)Jsml = —m n sin <j> n , 

(d y + M)/hl = -m n sm((j) n + a H + «i), 

. (d y + M)f 3L = m n cos(cf) n + «i), 

where we use cosai = \Jvn? n — M 2 /m n and sin oc\ = — M/m n . Then the KK mass 
condition will be 


(B.6) 


0 = 


cos f sm ( 


i cos 9 sin | cos(0 n + oti) 
—i sin 9 sin | cos (cf) n + ol\) 


i cos 6 sin | cos (cj) n — au) 
i sin 9 sin cos (<j) n — an) 
-(cos 2 9 cos | + sin 2 9 cos -4=) sin (f> ri 
- sin 9 cos 0(cos ^ — cos |) sin 0 n 


(B.7) 


—i sin 9 sin | cos(0 n + an) 


i cos 9 sin -j= cos(0 n + an) cos -j= sin <f> n 

— sin 9 cos 6*(cos — cos |) sin(0 n + «h + an) i sin 9 sin -j= cos (cf> n + an) 
— (sin 2 9 cos | + cos 2 9 cos ^=) sin(0 n + a H + «i) i cos 9 sin cos((p n + an) 


72 


(B.8) 
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= cos(0 n + «i) sin <p r , 


cos ((j) n + ai) = 0, 
( sin (j) n = 0, 

^4^ = sin 2 | - 

COS z OL\ 2 


— sm 


A 


A 


siiY — + [ shY —p= — sin* — ) sin 9 > cos ol\ 


V2 


(B.9) 


sm 2 | — sin 2 sin 2 9 = sin 2 | cos 2 9 + sin 2 sin 2 9. 


(B.10) 


We can see that the zeromode mass mo which mass is provided by the VEV v is 
corresponds to the third conditions Then we have the KK mass conditions as follows 

sin 2 ^ — fsm 2 ^ — sin 2 — 7 =] sin 2 9 = — - sin 2 — M 2 ) (B.ll) 

2 V 2 \/2 / rn n — M 

= ml-M* s ' mh2 ^ R V M2 ~ O- ( B - 12 ) 

The last expression corresponds to the case in M > m n , especially to the zero mode. 
The effects of mixture between the 3 and 6* reflects the left hand side of the above. 
Namely, if we set 9 —y 0, it recovers (B.3). On the other hand, if we set 9 —y tt /2 , 
the tsm is equivalent to the r in the 6* so that the yukawa coupling is give by 

sin 2 — = — mn sinh 2 (n Ri/M 2 — ml). (B.13) 

y/2 ml — M 2 

It corresponds to replace the g/2 in eq. (B.3) with g/^J 2. 
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